Adaptive User Models for Intelligent Information Filtering

As networked systems grow in size, the amount of data available to users has increased dramatically. The result is an information overload for the user. In this project, an intelligent information filtering system reduced the user's search burden by automatically eliminating incoming data predicted to be irrelevant. These predictions are learned by adapting an internal user model which is based upon user interactions. This report describes the information filtering problem and examines three techniques for filtering information: global hill climbing, genetic algorithms, and preliminary work with neural networks using radial basis functions. 1 The Information Overload Problem With the advent of networked systems, computer users are inundated with information that they cannot efficiently utilize. Tools are urgently needed to assist the user with information filtering devices in order to reduce the user's search burden. This project examined the Usenet News system as a testbed for the filtering algorithm. In the Usenet system, users throughout the world intermittently post articles to a common bulletin board. The number of articles posted may be very large; e.g., newsgroups may receive hundreds of articles daily. The goal is to predict whether new articles are likely to be of interest, or not of interest, based upon the prior behavior of the user. This is an extremely fuzzy and difficult problem to define because users are notorious for their inconsistency in their behavior patterns and changing interests. One of the difficult constraints imposed by this type of problem is the necessity for incrementality. Many learning algorithms, such as those based on neural networks, require repeated training epochs over a fixed data set. In the Usenet News problem, the data set is constantly changing as incoming messages are posted. To ensure consistency, the method would need to store all messages that were ever posted. When new messages arrive, the system would need to retain the old as well as the new messages. This is clearly undesirable due to the time requirements for training and the space required to store all messages. Many approaches to the information filtering problem bypass this problem by typically forcing the user to explicitly define what should be filtered, e.g. via a keyword-based database language [1].